Dispersion

The heatmap chart below shows the NPS responses dispersion.

On the heatmap chart above, there is a strong response concentration at the top of the chart – especially at level 10 and to a lesser extent 9 and 8 –, and there is some relatively dense concentration as well at the zero level.

At any level, months are not populated evenly, but at levels 10, 9, and 8 all months are populated in a relatively substantial measure. At level 10, the responses number culminates at 154 in March and never falls below 116 responses per month. At levels 9 and 8, the minimum per month is 45.

By the way, at level zero, the minimum per month is still 35. Other levels are less populated, and the responses number per month even falls in one case at a figure as modest as 6.

Content

Except for scores, what do available input data provide as information? We see it in the next interactive table.

In the table above, each NPS response is characterized among others by

• the market where the customer banks,
• the response day,
• the customer’s name,
• the score.

From the Kaggle website, we learn that the question asked was “how likely are you to recommend X to your friends and family?” and that “This a dataset from 2021, generated using real distributions of NPS data from a retail bank.”

Were there follow-up questions? We do not know. Would that have been useful? According to Jennifer Rowe, writing on the Zendesk website, “In fact, the most valuable aspect of your NPS survey results is the open-ended feedback that you receive from your customers. From this feedback, you can learn exactly what parts of the customer experience can use attention and improvement.”

We don’t know the very survey schedule either. For instance, is it a relational or a transactional survey? This difference is emphasized on the Qualtrics website: relational surveys are carried out on a regular basis while transactional surveys are sent after the customer has contacted the company.

Consequently, we have to focus on scores, markets, and timing.

Size

What is the survey size or rather the size of the sample we have access to since data are not the real ones but have been generated from the real distributions?

The number of responses is 5,000, which is definitely enough for the current exercise.

Is it representative of customer satisfaction on the various segments of the different markets involved over the whole year 2021? The importance of this issue is highlighted by Jennifer Rowe on the Zendesk website. But we do not have that kind of information.

The number of respondents is 4,829. Since this number is inferior to the number of responses, some respondents have responded more than once. Thanks to the presence of these multirespondents, we will be able to build cohorts of respondents, for example the cohort of respondents who responded in the first quarter and did it again in at least one other quarter.

Our approach will be divided into three steps:

• segmentation by markets,
• segmentation by periods,
• segmentation by cohorts.

Let us first dive into market segmentation.

Responses Count

The NPS responses total 5,000. How do they break down by market?

Responses are almost evenly broken down by market, namely approximately one third for each market, with the UK market representing one percentage point more than each of the other two markets. Consequently, the three markets contributed approximately with the same weight to the aggregate NPS of 12 reached by the three markets together (see the table just above).

The NPS responses are not the actual ones: they have just been generated from real distributions. We do not know the actual responses count by market and we cannot compare with recommendations such as Jennifer Rowes’s recommendations on the Zenkesk website.

NPS^® by Market

On the three markets, the retail bank reaches positive NPS results. With an NPS of 17, the MEX market is hovering above the other two markets. The US market is at 10 and the UK market at 8.

Are these results satisfactory?

Let us start with a general consideration issued by Bain & Company and reported by Cvetilena Gocheva on the CustomerGauge website: “Bain & Company (the creators of NPS) note that a good NPS score is 0 and above. Above 50 is excellent and above 80 is world class. … However, it’s important to compare yourself to others in your industry.”.

The three scores are positive, so, generally speaking, good. Let us hazard a comparison. Qualtrics published an average NPS of 24 for the banking industry in the US on the basis of a study conducted in 2021. A PDF can be downloaded from “Economics of NPS in the Banking Industry - Qualtrics https://www.qualtrics.com › uploads › 2022/04”. Compared to this standard, the average score of 10 obtained on the US market is below average. But is our dataset derived from a retail bank’s real distributions completely comparable with this average of 24?

Categories

The next graph panel breaks down NPS responses by market and by respondent category.

The MEX market has approximately the same number of passives as the UK market. The NPS superiority of the MEX market comes mainly from the detractors – exactly 100 detractors less on the MEX market – and secondarily from the promoters – 36 promoters more on the MEX market. This explains the difference in NPS: 17 on the MEX market, 8 on the UK market.

The US market has an NPS of 10, against 8 for the UK market. In fact, the US market has a few more promoters – 8 promoters more – and a few detractors less – 8 detractors less. The lower number of passives along the US side also explains one third of the difference between the two markets.

The next graph gives the percentage of promoters, passives, and detractors by market.

The MEX market performs better than the other two markets in both respondent categories that matter directly: higher percentage of promoters – with 48 % – and lower percentage of detractors – with 31 %.

The US market is almost as good in promoters but has a substantially higher percentage of detractors – namely 37 %.

All Score Levels

Let us have a look at the breakdown of responses for each of the 10 possible scores, on each market.

There is more concentration at the extremes on the MEX market if we just take into account the levels 0, 8, 9, and 10. Indeed, responses whose score is one of these figures total 76.4 % on the MEX market, 66.9 % on the UK market, and 70.1 % on the US market.

This could remind us of controversies about the criteria used to classify respondents into promoters, passives, and detractors. For instance, Alexander Dobronte published an article on CheckMarket: “Why there needs to be a European variant of the Net Promoter Score”. He argued that “In classic NPS scoring, the 8 from these respondents has no weight! They are ignored. That is why so many European companies have neutral NPS scores. What I propose is a European Net Promoter Score variant where an 8 also counts as a promoter and 6 as passive.”

Clifford Lewis and Michael Mehmet in SAGE Journals also raised questions, among others about the boundaries of the groups of promoters, passives, and detractors, providing a long list of references.

In another direction, Maurice FitzGerald nuances the origins of NPS differences between countries in an article titled “Does culture affect NPS / customer survey outcomes?”. He ends his article with: “Comparing country scores to each other mainly wastes time and delays getting on with improvements. Only removing the numbers from the scoring scales can help avoid your audience making irrelevant comparisons.”

In our case, NPS differences have been noticed between the three markets but we can hardly discriminate cultural factors from others since we lack the kind of extensive information Maurice FitzGerald had at his disposal. From his article, let us simply remember that comparisons between countries are tricky.

Now, let us dive into segmentation by periods.

Quarterly Evolution

On a quarter basis, we see

• on the MEX market: a continuous upward NPS trend;
• on the UK market: a slump in the second and third quarters followed by a rally that nearly canceled it;
• on the US market: a growth in the second quarter followed by a downward movement that caused NPS to fall well below the level of the first quarter, to an even slightly negative level of -1.

Let us retrieve more detailed information by working on a month basis.

Monthly Evolution

Switching from a quarter basis to a month basis brings more contrast at least for the MEX and the UK markets.

On the MEX market, the first six months and the last six months show rather different profiles. During the first six months, a sequence of up and down movements prevailed, which of course was not perceptible on a quarter basis. During the following six months, an upward trend took hold, with the notorious exception of November, during which NPS fell significantly; the fall of November was not perceptible either on a quarter basis.

On the UK market, there are three “grapes” of results: one for the first quarter, one for the second quarter, and one for the months August to October. This means that, for the first six months, the quarterly and the monthly profiles do not show substantial dissimilarities. But the downward movement of the second and third quarters is interrupted by a rebound in July, which did not show on a quarter basis; moreover, this downward movement extends beyond the third quarter into October, which did not show either on a quarter basis. The months August to October fell to a negative level, which did not appear either in quarterly figures. Last, the recovery noted in the fourth quarter is actually only seen in November and December once NPS is expressed by month.

On the US market, the inverted V profile observed on a quarter basis erodes on a month basis, producing a plateau from March to August, with the notable exception of July, which brings about a slump. The steady downtrend that shows in the third and fourth quarters shows with a lag on a month basis, namely from September to December.

Weekly Evolution

Let us dive some further into volatility by visualizing the NPS evolution on a week basis in the following graph panel.

To keep a vision of the general movement, to the weekly figures is added a smooth curve generated by the LOESS algorithm.

In the graph panel above, weeks are identified by numbers. Conversion to dates can easily be found on this website.

Monthly moves can correspond to continuous trends in the same direction at week level. Let us take an example on the UK market: NPS fell in April on a month basis; the first four weeks of April – that is to say weeks 13 (partially in April), 14, 15, 16 – went into the same direction with rather low NPS.

In other cases, weekly NPS can show up and downs and give indecisive indications about the month NPS. Let’s look at the example of October on the UK market. October has been one of the worst months with an NPS of -4, but, on a week basis, this contrasts with week 40 – the second week in October – being the top week of the year with an NPS of 38.

Decomposing results into respondent categories could bring some additional insights: this is done by the next graph panel.

On the US market, NPS has been rather low in July during weeks 27, 28, and 30. When looking at the graph panel above, causes become clearer: weeks 27 and 30 were marked by a relatively high number of detractors; in addition, weeks 28 and – once again – 30 were characterized by a limited number of promoters.

On the UK market, week 40 has been the top one in terms of NPS. The graph above indicates that it has been high in promoters and low in detractors.

After describing NPS evolution by quarter, month, and week, let us turn to cohort analysis, which will allow us to focus again on quarterly evolution but only among multirespondents who responded in different quarters.

NPS by Cohort

The next graph panel displays NPS for the cohort starting in the first quarter and for the cohort starting in the second quarter, on the three markets separately.

On the MEX market, the two cohorts follow similar directions. The first cohort starts at 13. In the second quarter, it crashes and drops to the minimum – that is to say -100. It then rebounds to the maximum – that is to say 100 – and ends at 25 – thus above the first quarter level. The second cohort begins at 15, jumps up to 100 and ends at 0 – thus below the join quarter.

On the UK market, the first cohort starts at 16, falls at -100, and then follows an upward movement up to 50. The second cohort shows an upward trend from 4 to 83.

On the US market, moves are mostly ascending and no negative NPS appears. Both cohorts finish at 100.

In a snapshot,

• all cohorts follow a positive path at least when comparing the end figure to the start figure, with the exception of the second cohort on the MEX market;
• movements are often ample and jumpy.

Since most NPS moves are so ample and jumpy, it would be interesting to know the responses number in each cohort after the join quarter in order to check whether NPS is representative after the join quarter.

So, let us compute the respondent retention rates by cohort and by market.

Retention

The three tables below show, each for one market, the retention rates of the three cohorts.

The situation is similar on the three markets with regard to the respondent retention rate: respondent retention is negligible.

The almost complete respondent churning can easily cause the instability noticed in the previous section in the NPS trends by cohort. Indeed, after the join quarters, NPS is calculated on numbers of respondents that are almost nil. Consequently, the NPS trends by cohort – most of which were globally on the upside – cannot be considered significant.

Why is respondent churning so high? Neither the input data nor the documentation provided provides a clue for further analysis.